Fully supervised salient object detection (SOD) has made considerable progress based on expensive and time-consuming data with pixel-wise annotations. Recently, to relieve the labeling burden while maintaining performance, some scribble-based SOD methods have been proposed. However, learning precise boundary details from scribble annotations that lack edge information is still difficult. In this paper, we propose to learn precise boundaries from our designed synthetic images and labels without introducing any extra auxiliary data. The synthetic image creates boundary information by inserting synthetic concave regions that simulate the real concave regions of salient objects. Furthermore, we propose a novel self-consistent framework that consists of a global integral branch (GIB) and a boundary-aware branch (BAB) to train a saliency detector. GIB aims to identify integral salient objects, whose input is the original image. BAB aims to help predict accurate boundaries, whose input is the synthetic image. These two branches are connected through a self-consistent loss to guide the saliency detector to predict precise boundaries while identifying salient objects. Experimental results on five benchmarks demonstrate that our method outperforms the state-of-the-art weakly supervised SOD methods and further narrows the gap with the fully supervised methods.
translated by 谷歌翻译
基于3DCNN,ConvlSTM或光流的先前方法在视频显着对象检测(VSOD)方面取得了巨大成功。但是,它们仍然遭受高计算成本或产生的显着图质量较差的困扰。为了解决这些问题,我们设计了一个基于时空存储器(STM)网络,该网络从相邻帧中提取当前帧的有用时间信息作为VSOD的时间分支。此外,以前的方法仅考虑无时间关联的单帧预测。结果,模型可能无法充分关注时间信息。因此,我们最初将框架间的对象运动预测引入VSOD。我们的模型遵循标准编码器 - 编码器体系结构。在编码阶段,我们通过使用电流及其相邻帧的高级功能来生成高级的时间特征。这种方法比基于光流的方法更有效。在解码阶段,我们提出了一种有效的空间和时间分支融合策略。高级特征的语义信息用于融合低级特征中的对象细节,然后逐步获得时空特征以重建显着性图。此外,受图像显着对象检测(ISOD)中常用的边界监督的启发,我们设计了一种运动感知损失,用于预测对象边界运动,并同时对VSOD和对象运动预测执行多任务学习,这可以进一步促进模型以提取提取的模型时空特征准确并保持对象完整性。在几个数据集上进行的广泛实验证明了我们方法的有效性,并且可以在某些数据集上实现最新指标。所提出的模型不需要光流或其他预处理,并且在推理过程中可以达到近100 fps的速度。
translated by 谷歌翻译
确定复杂系统背后的因果关系在不同领域(例如决策,政策实施和管理建议)中起着重要作用。但是,关于时间事件序列数据的现有因果关系研究主要集中于单个因果发现,这是无法利用合并因果关系的。为了填补在时间事件序列数据上发现发现的合并原因,消除和募集原则被定义以平衡因果组合的有效性和可控性。我们还基于反应点过程来利用Granger因果关系算法来描述实体之间的燃料或抑制行为模式。此外,我们设计了“电动电路”的信息性和美学视觉隐喻,以编码汇总因果关系,以确保我们的因果关系可视化是非重叠和不相互作用的。各种排序策略和聚合布局也嵌入了我们基于平行的,定向和加权的超图中,以说明合并因果关系。我们开发的合并因果关系视觉分析系统可以帮助用户有效地探索合并的原因以及个人原因。这种交互式系统支持多样化的订购策略以及重点和上下文技术,以帮助用户获得不同级别的信息抽象。通过进行试验用户研究和事件序列数据的两项案例研究,进一步评估了系统的有用性和有效性。
translated by 谷歌翻译
动态图可视化吸引了研究人员的集中度,因为它代表了多个领域的实体之间的时变关系(例如,社交媒体分析,学术合作分析,团队运动分析)。集成视觉分析方法对于呈现,比较和审查动态图是结果的。即使开发了多年的动态图可视化,但是如何有效地可视化具有微妙变化的大规模和时间密集型动态图数据对研究人员仍然具有挑战性。为了为此类动态图数据提供有效的分析方法,我们提出了一种快照生成算法,该算法涉及人类中的人类,以帮助用户将动态图分为多粒性和分层快照,以进一步分析。此外,我们设计了视觉分析原型系统(DGSVI),以帮助用户有效访问动态图见解。 DGSVI集成了图形操作接口,以帮助用户在视觉上和交互式上生成快照。它配备了可视化动态图数据的层次快照的概述和详细信息。为了说明我们提出的此类动态图数据的建议方法的可用性和效率,我们在竞争中介绍了基于篮球运动员网络的两个案例研究。此外,我们进行了评估,并收到经验丰富的可视化专家的激动人心的反馈。
translated by 谷歌翻译
我们为来自多视图立体声(MVS)城市场景的3D建筑物的实例分割了一部小说框架。与关注城市场景的语义分割的现有作品不同,即使它们安装在大型和不精确的3D表面模型中,这项工作的重点是检测和分割3D构建实例。通过添加高度图,首先将多视图RGB图像增强到RGBH图像,并且被分段以使用微调的2D实例分割神经网络获得所有屋顶实例。然后将来自不同的多视图图像的屋顶实例掩码被聚集到全局掩码中。我们的面具聚类占空间闭塞和重叠,可以消除多视图图像之间的分割歧义。基于这些全局掩码,3D屋顶实例由掩码背部投影分割,并通过Markov随机字段(MRF)优化扩展到整个建筑实例。定量评估和消融研究表明了该方法的所有主要步骤的有效性。提供了一种用于评估3D建筑模型的实例分割的数据集。据我们所知,它是一个在实例分割级别的3D城市建筑的第一个数据集。
translated by 谷歌翻译
扩散张量成像(DTI)已被用于研究神经退行性疾病对神经途径的影响,这可能导致这些疾病的更可靠和早期诊断,以及更好地了解它们如何影响大脑。我们介绍了一种基于标记为DTI光纤数据和相应统计数据的智能视觉分析系统,用于研究患者组。系统的AI增强界面通过组织和整体分析空间引导用户,包括统计特征空间,物理空间和不同组的患者的空间。我们使用自定义机器学习管道来帮助缩小此大型分析空间,然后通过一系列链接可视化务实拨动它。我们使用来自Parkinson进展标记倡议的研究数据库的实际数据进行多种案例研究。
translated by 谷歌翻译
This paper focuses on designing efficient models with low parameters and FLOPs for dense predictions. Even though CNN-based lightweight methods have achieved stunning results after years of research, trading-off model accuracy and constrained resources still need further improvements. This work rethinks the essential unity of efficient Inverted Residual Block in MobileNetv2 and effective Transformer in ViT, inductively abstracting a general concept of Meta-Mobile Block, and we argue that the specific instantiation is very important to model performance though sharing the same framework. Motivated by this phenomenon, we deduce a simple yet efficient modern \textbf{I}nverted \textbf{R}esidual \textbf{M}obile \textbf{B}lock (iRMB) for mobile applications, which absorbs CNN-like efficiency to model short-distance dependency and Transformer-like dynamic modeling capability to learn long-distance interactions. Furthermore, we design a ResNet-like 4-phase \textbf{E}fficient \textbf{MO}del (EMO) based only on a series of iRMBs for dense applications. Massive experiments on ImageNet-1K, COCO2017, and ADE20K benchmarks demonstrate the superiority of our EMO over state-of-the-art methods, \eg, our EMO-1M/2M/5M achieve 71.5, 75.1, and 78.4 Top-1 that surpass \textbf{SoTA} CNN-/Transformer-based models, while trading-off the model accuracy and efficiency well.
translated by 谷歌翻译
Decompilation aims to transform a low-level program language (LPL) (eg., binary file) into its functionally-equivalent high-level program language (HPL) (e.g., C/C++). It is a core technology in software security, especially in vulnerability discovery and malware analysis. In recent years, with the successful application of neural machine translation (NMT) models in natural language processing (NLP), researchers have tried to build neural decompilers by borrowing the idea of NMT. They formulate the decompilation process as a translation problem between LPL and HPL, aiming to reduce the human cost required to develop decompilation tools and improve their generalizability. However, state-of-the-art learning-based decompilers do not cope well with compiler-optimized binaries. Since real-world binaries are mostly compiler-optimized, decompilers that do not consider optimized binaries have limited practical significance. In this paper, we propose a novel learning-based approach named NeurDP, that targets compiler-optimized binaries. NeurDP uses a graph neural network (GNN) model to convert LPL to an intermediate representation (IR), which bridges the gap between source code and optimized binary. We also design an Optimized Translation Unit (OTU) to split functions into smaller code fragments for better translation performance. Evaluation results on datasets containing various types of statements show that NeurDP can decompile optimized binaries with 45.21% higher accuracy than state-of-the-art neural decompilation frameworks.
translated by 谷歌翻译
Image Virtual try-on aims at replacing the cloth on a personal image with a garment image (in-shop clothes), which has attracted increasing attention from the multimedia and computer vision communities. Prior methods successfully preserve the character of clothing images, however, occlusion remains a pernicious effect for realistic virtual try-on. In this work, we first present a comprehensive analysis of the occlusions and categorize them into two aspects: i) Inherent-Occlusion: the ghost of the former cloth still exists in the try-on image; ii) Acquired-Occlusion: the target cloth warps to the unreasonable body part. Based on the in-depth analysis, we find that the occlusions can be simulated by a novel semantically-guided mixup module, which can generate semantic-specific occluded images that work together with the try-on images to facilitate training a de-occlusion try-on (DOC-VTON) framework. Specifically, DOC-VTON first conducts a sharpened semantic parsing on the try-on person. Aided by semantics guidance and pose prior, various complexities of texture are selectively blending with human parts in a copy-and-paste manner. Then, the Generative Module (GM) is utilized to take charge of synthesizing the final try-on image and learning to de-occlusion jointly. In comparison to the state-of-the-art methods, DOC-VTON achieves better perceptual quality by reducing occlusion effects.
translated by 谷歌翻译
In recent years, the Transformer architecture has shown its superiority in the video-based person re-identification task. Inspired by video representation learning, these methods mainly focus on designing modules to extract informative spatial and temporal features. However, they are still limited in extracting local attributes and global identity information, which are critical for the person re-identification task. In this paper, we propose a novel Multi-Stage Spatial-Temporal Aggregation Transformer (MSTAT) with two novel designed proxy embedding modules to address the above issue. Specifically, MSTAT consists of three stages to encode the attribute-associated, the identity-associated, and the attribute-identity-associated information from the video clips, respectively, achieving the holistic perception of the input person. We combine the outputs of all the stages for the final identification. In practice, to save the computational cost, the Spatial-Temporal Aggregation (STA) modules are first adopted in each stage to conduct the self-attention operations along the spatial and temporal dimensions separately. We further introduce the Attribute-Aware and Identity-Aware Proxy embedding modules (AAP and IAP) to extract the informative and discriminative feature representations at different stages. All of them are realized by employing newly designed self-attention operations with specific meanings. Moreover, temporal patch shuffling is also introduced to further improve the robustness of the model. Extensive experimental results demonstrate the effectiveness of the proposed modules in extracting the informative and discriminative information from the videos, and illustrate the MSTAT can achieve state-of-the-art accuracies on various standard benchmarks.
translated by 谷歌翻译